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► Voice, facsimile, and electronic mail messag- 
ing is integrated in a system that converts e-mail 
messages into voice-and-fax messages. An e- 
mafl message (400) is parsed into voiceable, 
prose, segments (403) and non-voiceable, 
non-prose, segments (404). Prose segments are 
converted into voice message segments (503) 
via text-to-speech facilities. Each non-prose 
segment is converted into a fax segment (504). 
Voice pointers (505) to fax segments are inser- 
ted into the voice message, in places corre- 
sponding to the non-prose segments in the 
e-mail message. The voice file (500) and fax file 
(510) are then stored for subsequent delivery of 
the message as an integrated voice-and-fax 
message. Conversion of integrated voice-and- 
fax messages into e-mail messages is likewise 
envisioned. 
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Technical Field 

This invention relates generally to integrated 
message systems, and sp cif ically to voice-and-fax 
message systems and electronic mail systems. 

Background of the Invention 

Multi-media messaging involves integrating dis- 
crete messages or individual message components 
of different media, e.g., voice, text, video, and the 
ability to deliver the integrated messages through a 
single universal mailbox. 

The integration of voice and facsimile (fax) mes- 
sages is well known in the art For example, U.S. Pa- 
tent No. 5,008,926, discloses an integrated voice- 
and-facsimile store-and-forward message system 
that provides for the sending, storage, and delivery of 
voice messages, fax messages, fax messages anno- 
tated with voice messages, and voice messages an- 
notated with fax messages. The disclosed system ad- 
ditionally provides for display of fax messages on ter- 
minal screens, and the conversion of fax printed text 
messages into voice messages via text-to-speech 
conversion facilities. 

Electronic mail, or e-mail, systems are also well 
known in the art. They provide for the sending, stor- 
age, and delivery between computer terminals of dis- 
playable data -text and/or graphics-- messages. 

Integration of e-mail systems with voice-and-fax 
message systems is highly desirable. For example, it 
is becoming common for salesmen to work out of "mo- 
bile offices" —their cars— which are equipped normal- 
ly with telephones, often with fax machines, but gen- 
erally not with data terminals, yet the salesmen would 
like to have the capability of receiving in their mobile 
offices messages sent to them in any form -voice, 
fax, or e-mail. The integration of fax and e-mail mes- 
saging is readily accomplished through conversion of 
e-mail messages into fax messages for delivery to 
their recipients. But while fax machines are more like- 
ly to be available to mobile recipients of the messag- 
es than data terminals, they are not as readily avail- 
able as telephones. Therefore, it is often preferable to 
instead accomplish the integration of voice and e- 
mail messaging through conversion of e-mail mes- 
sages into voice messages for delivery to their reci- 
pients. This type of conversion is also readily accom- 
plished if the e-mail messages constitute text only. 

A problem arises, however, when e-mail messag- 
es constitute information other than text -such as di- 
agrams, charts, special symbols, computer code 
fragments, pictures, etc.- which is not suitable for 
conversion into a voice message. Attempts to convert 
such data message segments into speech result in 
unintelligible garble. This may be avoided by simply 
identifying and discarding the non-voiceable data 
message segments, but this results in the delivery of 



an incomplete message to the recipi nt -an unac- 
ceptable r suit 

Summary of the Inv ntion 

5 

This invention is directed to solving thes and 
other problems and disadvantages of the prior art. 
Generally according to the invention, data messag- 
ing, such as e-mail messaging, is integrated with au- 

10 dio-and-image messaging, such as voice-and-fax 
messaging, by converting voiceable segments of a 
data message into voice message segments, con- 
verting non-voiceable segments of the data message 
into image message segments, and substituting in 

15 the voice message voiced pointers to the image mes- 
sage segments in place of the non-voiceable seg- 
ments. Alternatively, the transformation of the mes- 
sage from one media to another may be done in the 
reverse direction. 

20 According to specific principles of the invention, 
segments of information (for example, of a received 
message) represented in a first medium (for example, 
e-mail) which can be represented in a second me- 
dium (for example, voice) are identified, representa- 

25 tion of the identified segments is convened from the 
first medium into the second medium, representation 
of other segments of the obtained information is con- 
verted from the first medium into a third medium (for 
example, fax) and then pointers represented in the 

30 second medium and pointing to the segments repre- 
sented in the third medium are interspersed among 
the segments represented in the second medium. 
The information represented in the second and third 
media may then be substituted for the information 

35 represented in the first medium. 

The conversion of the information from one set of 
media to another allows the information to be deliv- 
ered and presented to a user in a form selected by, 
and most suitable to, the user. Yet, the multi-media 

40 form of the message ensures that none of the infor- 
mational content of the message need be discarded 
during the conversion process. Thus, for example, a 
traveling sales representative is able to receive the 
voice portion of the converted e-mail message on his 

45 or her mobile telephone and decide from the contents 
of the voice portion whether to go to the effort of print- 
ing the associated fax component of the message. If 
the recipient decides to receive the entire message, 
he or she is able to receive the entire e-mail message 

so as an integrated voice-and-fax message on his or her 
mobile telephone and fax without loss of any of the in- 
formation carried by the e-mail message. Further- 
more, the pointers used in the multi-media message 
to point from one medium to the other ensure that the 

55 recipient can follow the logical flow of the original 
message in the multi-media message, and can re- 
ceive all the information contained in the original mes- 
sage in the proper order. 
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Preferably, head r information of a received e- 
mail message is convert d for delivery to a r cipient 
into a voice header. This allows th recipient to exam- 
ine and browse through headers of integrated voice- 
and-fax messages in the same manner as he or she 
does with headers of conventional voice messages. 
Further preferably, certain white-space segments of 
the e-mail message which carry no user information, 
are discarded during the conversion process. This in- 
creases the efficiency and compactness of the result- 
ing integrated voice-and-fax message. 

These and other advantages and features of the 
invention will become more apparent from the follow- 
ing description of an illustrative embodiment of the in- 
vention considered together with the drawing. 

Brief Description of the Drawing 

FIG. 1 is a block diagram of a multi-media inte- 
grated message system embodying an illustra- 
tive example of the invention; 
FIG. 2 is a flow diagram of message storing and 
delivery functions of an integrated voice-and- 
facsimile store-and-forward system of the multi- 
media integrated message system of FIG. 1; 
FIG. 3 is a flow diagram of an e-mail message to 
integrated voice-and-facsimile message conver- 
sion function of an e-mail system of the multi-me- 
dia integrated message system of FIG. 1; and 
FIG. 4 is a symbolic representation of the e-mail 
message to integrated voice-and-facsimile mes- 
sage conversion performed by the function of 
FIG. 3. 

Detailed Description 

FIG. 1 shows an illustrative telecommunications 
system. Jt comprises telephone terminals 29 and fax 
terminals 30 connected by a private branch exchange 
switch (PBX) 24 to a message system 1000. Message 
system 1000 stores voice and fax messages in a 
mass storage 1 2, such as a disk, operating under con- 
trol of a controller 10. Terminals 29 and 30 access 
message system 1000 through PBX 24 over phone 
lines 1004 and a voice/data interface 34 that interfac- 
es phone lines 1004 to the internal circuits and func- 
tions of message system 1 000, to store the messages 
in system 1000 and to retrieve the messages from 
system 1000. Message system 1000 is illustratively 
the AT&T Ovation .multimedia communication sys- 
tem, or an AT&T voice-mail system of the type dis- 
closed in EP-A-0554980 or in EP-A-0581484, this ap- 
plication, which systems are further adapted to store 
both voice and fax messages. Alternatively, portion 1 
of the system of FIG. 1 which includes terminals 29 
and 30, PBX 24, and message system 1000, is illus- 
tratively the integrated voic -and-facsimile store- 
and-forward system disclosed in U.S. Patent No. 
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5,008,926. 

Whil the operation of message system 1000 is 
described in the above-referenc d patent, a brief 
overview of its functionality relevant hereto is pre- 
5 sented here and in FIG. 2 for complet ness of de- 
scription. As mentioned above, message system 
1000 operat s under the guidance of a control 10. 
Control 10 is illustratively a general-purpose proces- 
sor operating under control of programs 1006 stored 
10 in its memory. Execution of programs 1006 imple- 
ments the store-and-forward messaging functionali- 
ty f lowcharted in FIG. 2. 

In response to receipt of a call, at step 200 of FIG. 
2, control 10 determines from caller input whether or 
15 not this is a message storage request, at step 202. If 
so, control 10 identifies from the caller's input the 
caller/message sender and the intended message re- 
cipient, at step 204, and the message medium or me- 
dia, at step 206. In this example, the media may be 
20 voice, fax, or voice-plus-fax, though other media, 
such as video, may also be allowed. Control 10 then 
forms a message header that includes information 
such as the sender's identification, date and time of 
receipt of the message, etc., and sets media I.D. flags 
25 in the header to indicate the identified message me- 
dia, at step 208. Control 10 then receives the seg- 
ment, if any, of the message which is in the voice me- 
dium, at step 210, stores the received segment in a 
file in mass storage 12, at step 212, and sets a voice 
30 file pointer in the message header to point to the stor- 
ed file, at step 214. Control 10 performs the same 
functions for the segment, if any, of the message 
which is in the fax medium, at steps 214-220. Control 
1 0 then stores the header in the intended message re- 
35 cipient's universal mailbox in mass storage 12, atstep 
222, and returns to step 202 to determine if the caller 
has further requests. 

If the caller's request is not to store a message, 
as determined at step 202, control 10 determines 
40 from the caller's input whether this is a message re- 
trieval request, at step 230. If not, control 10 contin- 
ues its operation with other conventional operations, 
not relevant hereto, at step 250. If so, control 10 iden- 
tifies the caller, at step 232, and then retrieves stored 
45 message headers from the caller's universal mailbox 
and sends them to the caller, at step 234. Control 10 
then determines from the caller's further input wheth- 
er the caller has selected a message for retrieval, at 
step 236. If not, control 10 returns to step 202; if so, 
so control 10 examines the media I.D. flags of the head- 
er of the selected message to determine the mes- 
sage's media, at step 238. Control 10 then retrieves 
the message segments that are in the identified me- 
dia using the file pointers of the message header, and 
55 sends the message segments to the caller, at steps 
240-242. Control 10 then returns to step 202. 

As shown in FIG. 1 , message system 1 000 is con- 
nected by a local area network (LAN) 1001 to an elec- 
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tronic mail (e-mail) network 1003. LAN 1001 is inter- 
fac d to the internal circuits and functions of mes- 
sage system 1000 by a LAN interface 84, in a conven- 
tional mann r such as is disclos d in the above- 
r f renced patent no. 5,008,926. 5 

E-mail network 1003 is a conventional n twork, 
comprising a plurality of communicatively intercon- 
nected computers including the e-mail computer 
1002. Computer 1002 couples data terminals 28 to 
network 1 003, by means of which users can send and 10 
retrieve e-mail messages in a conventional manner. 

According to the invention, however, an e-mail 
user who is also served by system portion 1 has the 
choice of receiving e-mail messages either in the con- 
ventional manner, or as integrated voice-and/or-fax 15 
messages through portion 1 . The user's choice is pro- 
grammed into the user's e-mail profile stored by com- 
puter 1002. If the user has chosen to receive e-mail 
messages through portion 1, computer 1002 exam- 
ines an e-mail message received for that user to de- 20 
termine which segments thereof can be converted 
into a voice message via text-to-speech functions, 
and which segments thereof are not amenable to this 
conversion. The convertible segments are converted 
into a voice message, and the nonconvertible seg- 25 
ments are converted into a fax message. Voice refer- 
ences pointing to the fax message are inserted into 
the voice message in place of the nonconvertible seg- 
ments. The voice and fax messages are then deliv- 
ered to message system 1 000 for storage and subse- 30 
quent delivery to the recipient in the conventional in- 
tegrated voice-and-fax messaging manner. 

According to the invention, in order to integrate e- 
mail messages into the integrated voice-and- 
facsimile store-and-forward system which forms por- 35 
tion 1, computer 1002 executes programs 1005 stor- 
ed in its memory that implement the functionality 
flowcharted in FIG. 3. 

In response to receiving an e-mail message for a 
user served by it, at step 300 of FIG. 3, computer 1 002 40 
identifies the recipient in a conventional manner, at 
step 302, and then examines the identified recipient's 
stored user profile, at step 304, to determine, at step 
306, whether the user wishes to have e-mail messag- 
es delivered in the conventional manner via terminal 45 
28 or as integrated voice-and-fax messages via 
phone and fax terminals 29 and 30 by way of message 
system 1000. If conventional delivery is indicated, 
computer 1002 proceeds to handle the received mes- 
sage in the conventional e-mail manner, at step 310. so 

If integrated voice-and-fax delivery is indicated, 
computer 1002 parses the message, at step 312, to 
separate it into segments classified by type. This is 
illustrated by the logical conversion diagram of FIG. 
4. An e-mail message 400 is an ASCII-formatf ile with 55 
a predefined header 401 followed by a sequence of 
lines of ASCII characters. Besides the header, the fol- 
lowing types of segment lines are defined: white- 



space 402, consisting of a blank line; prose 403, con- 
sisting of natural language or numerals interspaced 
with standard punctuation; and non- prose 404, which 
is anything else. Com put r 1002 classifies each line 
of the message into one of the above four types. The 
header is identified by sentinels predefined for this 
particular e-mail system (e.g., DATE:, FROM:, SUB- 
JECT:). Whitespace 402 is identified by a lone new- 
line or linefeed character. Prose 403 lines are identi- 
fied by strings of ASCII characters forming words, 
numbers, and conventional (natural-language) punc- 
tuation. Any lines not identified as header 401, white- 
space 402, or prose 403 are classified as non- prose 
404. These are, for example, lines containing graph- 
ics symbols, nonstandard characters (e.g., bar, angle 
bracket, ampersand, numerical signs, etc.), non-stan- 
dard punctuation or standard punctuation used in 
non-standard manner (e.g., a sequence of colons or 
semicolons), a string that is a mix of letters and nu- 
merals, etc. 

Having classified each line of the message ac- 
cording to type and thus separated the message into 
segments 401-404, computer 1002 now interprets 
the information of header segment 401 to create, via 
conventional text-to- speech technology, a voice mes- 
sage header 501 conveying that information, at step 
314 of FIG. 3. Computer 1002 then converts prose 
segments 403 into voice message segments 503, 
again using text-to-speech technology, at step 316. 
Computer 1002 ignores and discards whitespace 
segments 402 that are interspersed among prose 
segments 403, at step 318. Computer 1002 also con- 
verts each non-prose segment 404 into a fax segment 
504 (such as a fax page), at step 319. Illustratively, 
computer 1002 does this by first using the ASCII form 
of the non- prose segment to generate a PostScript 
file via a utility such as Nroff or Troff, and then con- 
verting the PostScriptf ile into standard G3fax format. 
Alternatively, the conversion from ASCII to G3fax bit- 
map is performed directly. Computer 1002 retains, as 
part of the fax, whitespace segments 402 that are sur- 
rounded by non-prose segments 404. 

Next, computer 1002 inserts into the voice mes- 
sage, at the places that were occupied in e-mail mes- 
sage 400 by the non-prose segments 404, voice poin- 
ters 505 to the appropriate fax segments 504, at step 
320. An example of a voice pointer 505 is "Refer to at- 
tached document section x". Computer 1002 then 
stores the voice message with the voice pointers into 
a voice file 500 in its memory, and stores the fax seg- 
ments into a fax file 510 in its memory, at step 322. 
E-mail message 400 has now been converted into the 
format of integrated voice-and-fax messages dealt 
with by system 1000, and computer 1002 transfers 
the voice file 500 and fax file 510 of the integrated 
voice-and-fax message 600 over LAN 1001 to mes- 
sage system 1000, at step 324, and ends its opera- 
tion, at step 326. Illustratively, the transfer over LAN 
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is performed by means of an applications program- 
mer interface (API) program communicating by 
means of a utility such as transfer control protocol/In- 
ternet protocol (TCIP/IP) remote procedure call 
(RPC) server f unction in LAN interface 84. Controller s 
10 of system 1000 treats LAN 1001 as a caller and 
stores th messag 600 in the manner shown in FIG. 
2. Then, upon demand, controller 10 presents mes- 
sage 600 as an integrated voice-and-fax message to 
the recipient user, also in the manner shown in FIG. w 
2. 

Of course, various changes and modifications to 
the illustrative embodiment described above will be 
apparent to those skilled in the art For example, the 
conversion of the e-mail message into an integrated 15 
voice-and-fax message may be performed at the 
message originator's computer, in a central conver- 
sion host computer in the e-mail system, or in mes- 
sage system 1000. Also, the conversion need not be 
limited to voice and fax media, but may involve other 20 
media such as video. Furthermore, the process may 
be reversed, whereby received integrated voice-and- 
fax messages are converted into e-mail messages for 
delivery. Further still, features may be added to im- 
prove the coherence and appearance of the output 25 
For example, sentinels may be included in the e-mail 
message to direct which components of the e-mail 
message are to be converted to fax irrespective of 
whether they are non-pose or prose components. Or, 
limits may be set on the minimum line size of prose 30 
segments that will be converted to voice format; 
smaller prose segments interspersed with non-prose 
segments will then be converted to fax. 

35 

Claims 

1. A method of media conversion, CHARACTER- 
ISED BY the steps of: 

identifying (312) segments (403) of ob- 40 
tained information represented in a first medium 
in which the information is presentable to a per- 
son and which can be represented in a second 
medium in which the information is presentable to 
a person; 45 

converting (316) representation of the 
identified segments from the first medium into 
the second medium; 

converting (319) representation of other 
segments (404) of the obtained information from so 
the first medium into a third medium in which the 
information is presentable to a person; and 

interspersing (320) pointers (505), repre- 
sented in the second medium and pointing to the 
segments represented in the third medium, 55 
among the segments represented in the second 
medium. 



2. The method of claim 1 CHARACTERISED IN 
THAT 

the step of identifying comprises the steps 

of 

obtaining (300) an e-mail message (400), 

and 

id ntifying (312) segments (403) of the 
obtained e-mail message that can be converted 
into voice message segments; 

the first step of converting comprises the 
step of 

converting (316) the identified segments 
into voice message segments (503); 

the second step of converting comprises 
the step of 

converting (319) other segments (404) of 
the e-mail message into fax message segments 
(504); and 

the step of interspersing comprises the 
steps of 

interspersing (320) voice pointers (505) to 
the fax message segments among the voice 
message segments, and 

delivering (324) the received message as 
an integrated voice-and-fax message (600) com- 
prising the fax message segments and the voice 
message segments interspersed with the voice 
pointers. 

3. The method of claim 2 CHARACTERISED IN 
THAT 

the step of converting other segments 
comprises the steps of: 

selecting (312) other segments of the e- 
mail message that are not to be discarded; 

converting (319) the selected segments 
into the fax message segments; and 

discarding (318) segments (402) of the ob- 
tained message other than the identified and the 
selected segments. 

4. The method of claim 1 CHARACTERISED IN 
THAT 

the step of identifying comprises the steps 

of 

receiving (300) an e-mail message, and 
parsing (312) the received message into 

header (401), whitespace (402), prose (403), and 

non-prose (404) segments; 

the first step of converting comprises the 

steps of 

converting (314) the header segment into 
a voice header segment (501), and 

converting (316) the prose segments into 
voice message segments (503); 

the second step of converting comprises 
the step of 

converting (319) the non-prose segments 
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into fax message segments (504); and 

the step of interspersing comprises the 
steps of 

inserting (320) among the voice message 
segments, at places occupied in the received - 5 
mail message by the non-prose s gments, voice 
pointers (505) to the fax message segments, and 

delivering (324) the received message to a 
recipient as an integrated voice-and-fax message 
(600) comprising a voice message that includes w 
the voice header segments, the voice message 
segments, and the inserted voice pointers, and 
further comprising a fax message that includes 
the fax segments. 

15 

5. The method of claim 4 CHARACTERISED BY the 
further step of: 

discarding (31 8) the whites pace segments 
(402) except those that are surrounded by the 
non-prose segments. 20 

6. A media-conversion arrangement CHARACTER- 
ISED BY 

means (1002,1005:312) for identifying 
segments (403) of obtained information repre- 25 
sented in a first medium in which the information 
is presentable to a person and which can be rep- 
resented in a second medium in which the infor- 
mation is presentable to a person; 

processor means (1002,1005:316) for 30 
converting representation of the identified seg- 
ments from the first medium into the second me- 
dium; 

processor means (1002,1005:319) for 
converting representation of other segments 35 
(404) of the obtained information from the first 
medium into a third medium in which the informa- 
tion is presentable to a person; and 

means (1002,1005:320) for interspersing 
pointers (505), represented in the second me- 40 
dium and pointing to the segments represented in 
the third medium, among the segments repre- 
sented in the second medium. 

7. The arrangement of claim 6 CHARACTERISED 45 
IN THAT 

the identifying means comprise 
means (1 002. 1 005: 300) for obtaining an e- 
mail message, and 

means (1002,1005:312) for identifying so 
segments (403) of the obtained e-mail message 
that can be converted into voice message seg- 
ments; 

the first processor means comprise 
means (1 002,1 005:31 6) for converting the 55 
identified segments into voice message seg- 
ments (503); 

the second processor means comprise 



means (1002,1005:319) for conv rting 
other segments (401) of the e-mail message into 
fax message s gments (504); and 

the interspersing m ans comprise 

means (1002,1005:320) for interspersing 
voice pointers (505) to the fax message seg- 
ments among the voice message segments, and 

means (1002,1005:324) for delivering the 
received message as an integrated voice-and-fax 
. message (600) comprising the fax message seg-' 
ments and the voice message segments inter- 
spersed with the voice pointers. 

8. The arrangement of claim 7 CHARACTERISED 
IN THAT 

the means for converting other segments 
comprise: 

means (1002,1005:312) for selecting 
other segments of the e-mail message that are 
not to be discarded; 

means (1002,1005:319) for converting the 
selected segments into the fax message seg- 
ments; and 

means (1002,1005:318) for discarding 
segments (402) of the obtained message other 
than the identified and the selected segments. 

9. The arrangement of claim 6 CHARACTERISED 
IN THAT 

the identifying means comprise 
means (1002,1005:300) for receiving on c 
mail message, and 

processing means (1002,1005:312) for 
parsing the received message into header (401), 
whitespace (402), prose (403), and non-prose 
(404) segments; 

the first processor means comprise 
processing means (1002,1005:314) for 
converting the header segment into a voice head- 
er segment (501), and 

processing means (1002,1005:316) for 
converting the prose segments into voice mes- 
sage segments (503); 

the second processor means comprise 
processing means (1002,1005:319) for 
converting the non-prose segments into fax mes- 
sage segments (504); and 

the interspersing means comprise 
means (1002,1005:320) for inserting 
among the voice message segments, at places 
occupied in the received e-mail message by the 
non-prose segments, voice pointers (505) to the 
fax message segments, and 

means (1002,1005:324) for delivering the 
received message to a recipient as an integrated 
voice-and-fax message (600) comprising a voice 
message that includes the voice header seg- 
ments, the voice message segments, and the in- 
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serted voice pointers, and further comprising a 
fax messag that includes the fax segments. 

10. Th arrangement of claim 9 CHARACTERISED 
FURTHER BY 

means (1002,1005:318) for discarding th 
whitespace segments (402) except those that are 
surrounded by the non-prose segments. 
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FIG. 2 
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(57) Voice, facsimile, and electronic mail messag- 
ing is integrated in a system that converts e-mail 
messages into voice-and-fax messages. An e- 
mail message (400) is parsed into voiceable, 
prose, segments (403) and non-voiceable, 
non-prose, segments (404). Prose segments are 
converted into voice message segments (503) 
via text-to-speech facilities. Each non-prose 
segment is converted into a fax segment (504). 
Voice pointers (505) to fax segments are inser- 
ted into the voice message, in places corre- 
sponding to the non-prose segments in the 
e-mail message. The voice file (500) and fax file 
(510) are then stored for subsequent delivery of 
the message as an integrated voice-and-fax 
message. Conversion of integrated voice-and- 
fax messages into e-mail messages is likewise 
envisioned. 
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